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Abstract 

Assessment is too often concerned with measurement, rather than learning; however, there is a growing 
interest in research into formative assessment, which appears justified by studies into its effects on 
learning. Changes in higher education have led to increased numbers of students, many of whom are from 
non-traditional backgrounds. This has highlighted the need for transparency and student involvement in 
assessment. However, the corresponding pressures on staff and on resources mean that many desirable 
innovations are not easy to implement. The overall aim of this formative feedback intervention (FFI) was to 
provide timely and helpful feedback to international students who are final-year direct entrants in a large 
business school. Timeliness of feedback and the development of academic literacy were key concerns. The 
study concludes that although the FFI did not have a significant impact on module grades, the intervention 
was successful in getting students to engage in academic writing at an early stage. Most respondents 
perceived the feedback to be helpful and the feedback messages were clearly received and internalised. 
Whether appropriate actions were taken by the students to close the gap between their current and their 
target level requires further investigation. 
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Introduction 

Given the importance of assessment to almost everyone in education, it is to be expected that it is the 
subject of frequent debate. Though as Gibbs and Simpson (2004) point out, much of the attention it 
receives is negative, often used to support claims of falling standards, disputed grades and examiner 
incompetence. When the issue of assessment is raised in the media or quality assurance settings, the focus 
is likely to be on measurement, rather than on learning, which is the concern of this study and others 
mentioned within it. Gibbs and Simpson (2004) note that for a long time assessment in higher education 
was characterised by a lack of transparency, and founded on tacit knowledge, "that which we know but 
cannot tell," (Polyani, cited by Elander, 2003:117). So the move over the last decade towards assessment 
for learning, or even assessment as learning (Boyd & Bloxham, 2009), which encourages student 
involvement and engagement in the process, is to be welcomed. 

Perhaps due to initiatives associated with Assessment for Learning, there is a considerable amount of 
research into formative assessment (see McDowell, Sambell & Davison, 2009, for an overview) which 
appears justified by studies into its effects on learning. Black and Wiliam's (1998) review revealed 
significant and consistent positive effects of formative feedback on student learning across a wide range of 
educational contexts. Similarly, Hattie (1987, cited by Gibbs & Simpson, 2004) reported feedback as having 
the single most powerful effect on student achievement. The work of Sadler (1989) underpins much of the 
research; he identifies three necessary conditions for students to benefit. Students must: 

• possess a concept of the standard, or goal being aimed for 

• be able to compare their own work with this standard 

• engage in appropriate action which leads to some closure of the gap (Sadler, 1989:121). 
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To take these actions, Sadler argues, students must necessarily have some of the evaluative skills of their 
teacher, and this can by no means be taken for granted. 

Effective formative feedback not only gives useful information to students, but also to teachers, who 
can inform and shape teaching (Gibbs & Simpson, 2004; Nicol and Macfarlane-Dick, 2006). These 
authors provide very useful analyses of formative assessment and the conditions under which it promotes 
worthwhile learning. Both models are offered to teachers as a means to evaluate their own assessment 
practice, and were considered when designing the FFI discussed here. 

Changes in higher education have led to increased numbers of students, many of whom are from non- 
traditional backgrounds. This has highlighted the need for reform, though the corresponding pressures 
on staff and on resources mean that many desirable innovations are not easy to implement (Gibbs & 
Simpson, 2004). The many problems associated with feedback, led them to conclude that it is 'not a 
pretty picture' (Gibbs & Simpson, 2004:11). It would seem that there are barriers to a successful feedback 
dialogue from both student and staff perspectives, with both groups expressing frustrations. To begin with, 
there is evidence, and plenty of anecdotes, to suggest that some students do not, in fact, read feedback 
(Wojtas, 1998, cited by Duncan, 2007). This has led to suggestions that students today are instrumental 
and grade-oriented, and cited as evidence of a new type of student-consumer, an inevitable product 
of the modern educational marketplace. However, Higgins et al. (2002) coined the term 'conscientious 
consumer', finding that students were intrinsically motivated and sought feedback to achieve 'deep 
learning'. The reasons for the lack of interest in feedback cited by students in Higgins' study include: 

• poor quality 

• inconsistency 

• vague or difficult language that made it difficult to understand - that is if they could even read 
the handwriting. 

According to Lea and Street (1998) some inconsistency experienced by students may be explained by the 
modularised system, which means that they are moving between different discourses and responding to 
different discipline-specific expectations. They found that academics' efforts to help students to deal with 
this have failed to scratch below the surface, where criticisms of poor writing skills mask discipline-specific 
epistemological beliefs that many staff find hard to articulate. Furthermore, many assessment practices do 
not take adequate account of the issues of identity and power relationships in which they are embedded. 
Lea and Street's (1998) Academic Literacies Approach takes account of the 'cultural and contextual 
component of writing and reading practices' (1998:157). Building on this, Sutton and Gill (2010) use the 
term 'feedback literacy' arguing that: 

'Acquiring feedback literacy is part of the broader process of acquiring academic literacy, of 
learning to think, read and write in new ways' (2010:11). 

Many of the respondents in the studies mentioned here are those whose first language is English, so we 
might imagine the difficulty international students have in trying to come to terms with these issues. 

The importance of formative assessment means that we must appreciate the complexities involved in the 
process for both students and staff, for as Crisp (2007) notes, simply blaming the students is not enough. 
Barriers to effective communication might include: 

• increased workloads 

• high student-staff ratios 

• the number of students from non-traditional backgrounds, especially those for whom English is 
often an additional language. 

Despite student criticisms, many staff spend considerable time and effort writing student feedback. 

The problems with assessment outlined above have resonance with the authors of the present study. We 
sought to improve the learning and achievement of final-year direct-entrant undergraduates in a business 
school in the North of England by introducing formative assessment early in the academic year. In this 
context, timeliness and the development of academic literacy were key concerns. The study comes from 
a social constructionist perspective. Feedback messages are not assumed to be simply transmitted from 
teacher to student. They are jointly created between student and teacher, and are complex and difficult to 
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decipher (Nicol and Macfarlane-Dick, 2006). Furthermore, feedback messages are socially situated within 
power relations of learning and teaching and struggle for identity (Sutton & Gill, 2010) and within a wider 
unequal world order (Altbach, 2004). 

Aims and objectives 

This pilot study aims to evaluate a formative feedback intervention whose aim was to provide timely and 
helpful feedback to students so that they should be: 

• better prepared and more confident for semester one assignments 

• on track to meet the learning outcomes of the target module in semester two. 

The FFI was trialled by the authors with their students during the academic year 2009-10, with the 
expectation that if it proved successful, it would be adopted on a module-wide basis the following year. A 
third aim was to gather information for academic staff to inform the curriculum, both on the Professional 
Development Project (PDP) module, and in ASk (Academic Skills for international students) workshops. 
These workshops, delivered by Lecturers in English for Academic Purposes (EAP), are designed to support 
international students with English language and academic skills. 

The research attempted to answer the following questions: 

1. Using Gibbs' (1998) reflective cycle, what were the students' reflections on the usefulness of the FFI, 
both in terms of their understanding of their current level and how to "close the gap"? 

2. Did the FFI increase the students' confidence in preparing for semester one assignments? 

3. How useful were the ASk classes in preparing students for the FFI? What further support do they need? 

4. What impact did the FFI intervention have on the target module grades, and overall degree 
classifications of the cohort? 

Background 

The authors of this study come from two different disciplines: Accounting, and English for academic 
purposes, and are working together to enhance the learning of the students in one particular module: the 
PDP. They are also concerned with the development of the students' academic literacy and transferable 
skills. 

The PDP was developed for final-year direct-entrant students as an alternative to the traditional 
dissertation, due to the low marks attained on the latter. Most of these students are international, with 
most originating from South East Asia. It is a 30-credit module running over two semesters, whose mark 
represents 25% of the final degree classification. The assignment comprises two parts: 

• Part A is a reflective piece of writing entitled "Who am I as a learner?" 

• Part B is a project whose title is determined by the students themselves, requiring a literature 
review, though not necessarily primary research. 

The link between the two is the emphasis on the students' chosen career path, which is justified by the 
reflection in Part A. and which ideally underpins the choice of topic in Part B. 

Problems with the previous feedback model 

The problems we perceived and which prompted our intervention were that the students did not appear 
to have much prior experience of many of the academic skills required by this and other modules. We are 
aware that many assumptions are made with regard to international students and that the suggestion that 
they are 'lacking' could be construed as conforming to the 'deficit' approach (Wingate, 2006). Perhaps a 
fairer explanation is that some of the skills that international students have found work for them in their 
home institutions are valued less in their UK institutions (Volet, 1999). 

Our observation that many students needed further development of academic literacy was backed up by 
our earlier investigations into the prior learning experiences of previous cohorts from the same partner 
institutions. These showed that many had little or no experience of independent research, extended 
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academic writing and critical evaluation of source material (Burns & Foo, 2011). We are working to embed 
academic literacy within the discipline. Although we are cognisant of the tendency of the study skills 
approach' and even 'academic socialisation' (Lea & Street, 1998) to be unidirectional and teacher-led, time 
is a real constraint in the case of final-year direct-entrant students. 

Significantly, students the previous year had expressed concern at the timing of formal feedback, none of 
which came in the first semester. Their semester one assignments were done 'in the dark' with the same 
skills deficit being punished in all assignments. Feedback typically arrived some three weeks into semester 
two (Higgins et al., 2002) and when it was disappointing had a devastating effect on the self-belief and 
motivation of some individuals (Nicol and Macfarlane-Dick, 2006). Another key driver of the FFI was our 
own observation that students had difficulty choosing a topic for Part B of the project. This was being left 
until well into semester two, and as a result students were under great stress to find relevant literature. 
Advice from members of the teaching team, such as the need to source relevant theory to underpin 
examples and discussion of business practice was met by apparent lack of understanding. 

The intervention (FFI) 

Our response to this was to design a formative feedback intervention which would give feedback on 
a small proportion of the assignment, but which would involve academic skills that could, in theory, 
be used across a range of other modules. Common perceived weaknesses in student work included a 
lack of relevant literature, poor referencing, poor academic writing and a lack of critical evaluation. The 
intervention required students to write a mini literature review (up to 1,000 words) on the topic they were 
planning to use for Part B of the project. It was felt that this task would allow feedback on the students' 
understanding and ability to write a literature review, and it would oblige students to find a topic at an 
early date. The mini literature review was to be handed in week 8 of the first semester. 

The ASk workshops were run in tandem with PDP workshops, and were designed to provide practice in the 
key skills up to the handing in of the mini literature review. Following the feedback exercise, both ASk and 
PDP workshops could be adapted to respond to any issues arising. The submission of the mini literature 
review was felt to be a useful goal and was used to promote engagement with the ASk programme, which 
is neither strictly compulsory nor credit-bearing. 

Modes of feedback 

The PDP is one of the largest modules in the business school, with 218 students enrolled in the pilot study 
year, and with a teaching team of up to ten members. So it was important for the intervention to be 
practical and not to impinge on staff workloads. Students are not given an individual supervisor but instead 
are encouraged to work in learning sets in two-hour workshops. It was decided that feedback would be 
given during workshop time to avoid additional resourcing. From a pedagogical perspective, the one-to- 
one dialogue between student and teacher is highly valued by students (Murtagh and Baker, 2010). 

A matrix based on the school-wide marking criteria (McKeever et al., 2010) was adapted and extended for 
students whose first language was not English, and was used to indicate the current level of the student's 
work. The matrix, as seen in Figure 1., shows students where they are - as evidenced by their submitted 
mini literature review - according to a number of criteria. It identifies the skills needed to reach the 
next level, i.e. the 'gap' referred to by Sadler (1989). The annotated copy of the student work was used 
alongside the matrix to show examples of the issues under discussion. 

A further advantage of this approach was the opportunity it gave to engage students with the marking 
criteria used across the school. It would appear that this is rarely consulted, perhaps due to its opaque 
language (Murphy & Cornell, 2010). Engagement with the criteria is necessary if students are to develop 
the evaluative skills necessary to become self-regulated learners (Nicol and McFarlane-Dick, 2006) and to 
be able to close the gap. 

The decision to use the matrix aligned to the school criteria meant that, in addition to the feedback, a 
grade was also given, since otherwise it could have easily been inferred. This caused some deliberation 
on our part, since there is evidence that if a grade is given, less attention is given by students to the 
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comments. Furthermore, giving a grade is likely to lead to ego involvement on the part of the student, 
and can encourage competition rather than co-operation (see Butler, 1988 and Butler and Winnie, 1995, 
both studies cited by Nicol & Macfarlane-Dick, 2006). Although the assignment was an individual one, the 
authors sought to promote a co-operative environment where feedback was discussed and shared. 

Negative marks also run the risk of demotivating students as we had discovered with our own students in 
previous years. However, these findings seem to depend on the context and the individuals involved. It is 
suggested that if grades are task-focussed rather than people-focussed, and care is taken to highlight what 
students can do to improve their work, grades can be helpful (Black and Wiliam, 1998). 
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Figure 1 . FFI criteria matrix (extract) 


Data collection 

The data was collected in two stages: 

• Stage 1 was to meet research objective 1 

• Stage 2 was to meet research objectives 2 and 3 (see above). 

Objective 4 consisted of an analysis of the module grades and degree classifications. 


Stage 7 

Following the one-to-one feedback dialogue, a qualitative questionnaire, previously used by McKeever et 
al. (2010) was offered to students to gauge the perceived helpfulness of the feedback. Reflections were 
elicited using an adaptation of Gibbs' (1998) reflective cycle, the objective being to monitor: 

• students' immediate reactions to the feedback, i.e. their "gut feelings" following the tutorial 

• students' evaluation 2-3 days later 

• students' analysis and actions taken in response to the feedback beyond 2-3 days. 

The questionnaire was handed in approximately three weeks after it was issued, and 33 students 
responded. This may seem a low response rate, but it can probably be explained by the fact that 
participation was purely voluntary, and out of the large teaching team, besides the authors, only one or 
two members of staff actively encouraged their groups to take part. 
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Stage 2 

A further structured questionnaire comprising Likert scale responses to questions was offered 5 weeks after 
submission of the first questionnaire. This was at the beginning of semester 2, crucially after the students 
had completed their semester 1 assignments but before receiving their grades. 

Stage 2 yielded 50 responses. The questionnaire, with background information to the study and informed 
consent form, were posted electronically to the students with an invitation to participate, assuring them of 
confidentiality and anonymity, in accordance with the University's ethical guidelines. 

Results and discussion 

The Stage 1 questions were subjected to thematic analysis (Braun and Clark, 2006, cited by Murphy 
& Cornell, 2010). The salient themes for each group are presented below along with some example 
quotations. 

Stage 1 

Responses for the students' initial feelings provoked a range of emotional responses. Six students expressed 
disappointment with their performance and the grade received: 

...upset and nervous... 

...shocked my proposal was evaluated so low... 

...quite good but not as good as expectations... 

However, the majority (15) declared themselves happy: 

...glad, higher than expected...earlier worried...now relieved... 

A survey of the students' evaluation 2-3 days after the tutorial revealed a range of positive responses to 
the feedback. Nine students expressed acceptance of the feedback, four claimed that it had given them 
direction, two that it gave good advice, three mentioned specific areas for improvement, five that it was 
useful and ten 'other'. 

It was good ...to remind me the mistakes in reference. I cannot correct myself without... 
feedback 

I thought the feedback is good for me improving and providing a direction for subsequent work 
I need to improve my analysis deeper as well as my structure. 

I read my work and feedback again... I feel the mark given is higher than my work deserves. 

It's good in terms of point out the range of that my work is in and where to improve for a higher 
marks. 

I think the feedback is true. My work still has many shortcomings... 

I was graded 2(H) but my ultimate aim is first class or 2(i) and I believe I can achieve that now after 
the feedback. 

The responses for the students' analysis beyond 2-3 days did not change greatly: 

I should enrich the theoretical framework 

I have good referencing but it is not enough. I should get more journals related to topic 

As an overseas student, pay attention to the grammar 

I've learnt to use sound literature in order to support my arguments 

According to the questionnaire, actions taken by students beyond 2-3 days fall evenly across the FFI matrix 
criteria, based on areas they have identified to improve: 

• theory and practice recognition 

• using resources and referencing 

• presentation, structure and language. 
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Finding more journals to support... my analysis. 

. ..reference my work correctly and better... 

.. .practice in grammar and spelling... improve my reading and writing skill. 

.. .put more effort to research and evaluate the theories and models to find the best and most suitable. 
Spelling and grammar errors reduced through thinking in English logic and do more practice 


Table 1. Research Question 1 : Responses to the mode of delivery of feedback. 


Mode of 
feedback 

Not 

useful 

Slightly 

useful 

Useful 

Very 

useful 

Extremely 

useful 

Not 

disclosed 

Total 

number of 
respondents 

Written 

2 

4 

19 

16 

8 

1 

50 


(4%) 

(8%) 

(38%) 

(32%) 

(16%) 

(2%) 

(100%) 

Oral 

1 

6 

12 

22 

8 

1 

50 


(2%) 

(12%) 

(24%) 

(44%) 

(16%) 

(2%) 

(100%) 


The findings in Table 1. are in keeping with previous studies in which the learning dialogue is highly valued 
(Murtagh and Baker, 2010:25, Nicol and Macfarlane-Dick, 2006). 


Stage 2 

There was a striking similarity between the students' evaluation of the feedback and their actions taken in 
response to it 2-3 days after the tutorial. This emerged again in Stage 2 and is illustrated in the Wordles 
below (Figures 2 and 3) which show the students' response to the Stage 2 questions: 

• As a result of FFI what aspects of your study did it draw your attention to? 

• As a result of FFI what actions did you take in preparation for your semester 1 assignments? 
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Figure 2. Wordle of responses to Stage 
2 Question: As a result of FFI what 
aspects of your study did it draw your 
attention to? 
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Figure 3. Wordle of responses to Stage 
2 question: As a result of FFI what 
actions did you take in preparation for 
your semester 1 assignments? 


A clearer distinction between feedback analysis and actions taken had been expected. For example, to the 
latter question some specific actions such as signing up for a class in information literacy, or consulting a 
specific grammar book were, perhaps naively on the part of the staff, expected. With reference to Gibbs 
and Simpson's (2004:25) condition 9 - Feedback is received and attended to - we could conclude here 
that it had been received but whether it had been attended to, or acted upon could not be established. 

In other words, we could say that students appear to possess an understanding of the standard, but we 
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cannot say with confidence that the students knew how to close the gap (Sadler, 1989). Herein perhaps 
lies the difference between feedback and feed-forward (Boyd & Bloxham, 2009). Another explanation is 
that students act upon feedback messages intuitively rather than directly or instrumentally (Higgins et al., 
2006). As noted earlier, feedback messages are inherently complex. Further research is needed to clarify 
this point, perhaps in focus groups as these are useful for understanding multiple perspectives (Murphy and 
Cornell, 2010). 


Table 2. Research Question 2: Did the FFI increase the students' confidence in preparing for semester one 
assignments? 


Student 

Down 

Down a 

No 

Up a 

Up a 

Not 

Total 

level of 
confidence 

a lot 

little 

Change 

little 

lot 

disclosed 

number of 
respondents 

Written 

1 

(2%) 

4 

(8%) 

8 

(16%) 

28 

(56%) 

4 

(8%) 

5 

(10%) 

50 


Table 2. shows that nearly two-thirds of respondents reported an increase in self-confidence towards 
semester 1 assignments as a result of taking part in FFI. This provides support for the notion that feedback 
can be empowering for students (Nicol & Macfarlane-Dick, 2006). 


Table 3. Research Question 3: How useful were the ASk classes in preparing students for the FFI? 


How well did ASk classes prepare 
you for the FFI? 

Number of 
respondents to 
question 

Did not prepare me 

2 

(4%) 

Prepared me slightly 

8 

(16%) 

Prepared me 

24 

(48%) 

Prepared me well 

7 

(14%) 

Prepared me very well 

3 

(6%) 

Not disclosed 

6 

(12%) 

Total number of respondents 

50 

(100%) 


According to Table 3., just under half of the respondents felt that ASk classes had prepared them for the 
FFI, with a further 20% stating that ASk had prepared them well or very well. Yet, most of the feedback 
focussed on the need to develop academic skills rather than subject knowledge (see Wordles, Figures 2 and 
3). This confirms our earlier research into the students' lack of prior experience of independent research, 
extended academic writing and referencing of source material (Burns & Foo, 2011). 
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Table 4. Research Question 4: What impact did the FFI have on the target module grades, and overall 
degree classifications of the cohort? 



2007/08 

% 

2008/09 

% 

2009/10 

% 

2010/11 

% 

Firsts 

3 

00.3 

18 

0.07 

14 

0.09 

10 

0.05 

2.1 

18 

0.18 

100 

0.39 

46 

0.31 

78 

0.39 

2.2 

46 

0.46 

105 

0.41 

60 

0.40 

88 

0.44 

Thirds 

23 

0.23 

14 

0.06 

16 

0.11 

12 

0.06 

Pass 

10 

0.10 

17 

0.07 

13 

0.098 

10 

0.05 


100 


254 


149 


198 


Average 

project 

grade 55.84% 


54.89% 


59.46% 


60.14% 



As shown in Table 4., from 2009/10 to 2010/11 the average project grade increased on average by 1% 
per annum. While no statistical significance can be shown between FFI and grades, the qualitative data 
suggests the FFI has been beneficial. 


Conclusions 

From the evidence presented, we cannot conclude that the FFI impacted on module grades, though the 
average grade continues to improve. FFI appears to have been successful in engaging participating students 
in the writing of the extended project at an early stage. Most respondents perceived the feedback to be 
helpful; the feedback messages were clearly received and internalised, but whether appropriate actions 
were taken to close the gap requires further investigation. Both methods of feedback delivery were useful 
from the student perspective, enabling them to understand their current level, and what was needed to 
move beyond that, as well as potentially opening up a learning dialogue between student and teacher. 

From the authors' point of view, the matrix allowed feedback to be given consistently across a number of 
criteria without impinging too heavily on workloads, which was a key concern in a large module such as 
this. 

According to respondents, the FFI gave them direction and increased their confidence in preparing for 
other assignments. The feedback gave useful, though not unexpected feedback to staff, both in the 
discipline and ASk support. It also gave the ASk workshops a relevant context and goal, which appeared to 
increase student engagement. 

Clearly, more needs to be done to support the development of the academic literacy of this particular 
cohort, and, given the timescale available to direct-entrant students, it is tempting to suggest that they 
need more ASk support. However, Lea and Street (1998) show that a "bolt-on" approach is unlikely 
to provide a satisfactory solution, though collaborative projects such as this one which seek to embed 
academic literacy within the discipline may be more effective. The inclusion of self- and peer-assessment 
opportunities may be beneficial; ASk can support this, but the time it takes for students to develop their 
evaluative skills should not be underestimated. 

Learning about assessment practices is, arguably, as important as learning about the subject. We suggest 
that interventions such as this one are a step in the right direction, as they: 

• provide the opportunity for students to engage in feedback discourses 

• allow learning dialogues to take place in low-risk settings 

• make assessment processes and criteria transparent. 
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